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G radually Internet-based educational resources 
are making their way into the school mathe- 
matics curriculum (Handal & Herrington, 
2003). Online resources are potentially useful 
compared to normal courseware because of their 
abundance, availability at no cost, platform-free acces- 
sibility, and their wide reaching accessibility. On the 
other hand, a major limitation of online resources is 
their lack of appropriate pedagogy, coupled with poor 
instructional design and layout. According to Alessi 
and Trollip (2001, p. 392), “The tendency for the Web 
to be used only for presentation of materials greatly 
restricts its instructional potential”. 


provide practical information 

Evaluating courseware 

about some key websites. 

How do we know that courseware is well designed 
and pedagogically sound? There are at least two 
approaches in the evaluation of courseware. The first 
approach makes use of evaluation forms and check- 
lists that assess mostly interface design, navigation 
and/or control features of courseware as well as other 
intertwined pedagogical variables. These features are 
then compared against a set of ideal criteria appro- 
priate from an instructional design point of view. A 
number of evaluation forms and checklists have been 
designed in this way (e.g., Alessi & Trollip, 1991; 
Reeves & Harmon, 1994). A second type of evaluation 
is referred to as context-based evaluation since assess- 
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ment is carried out as the resource is used by the 
learner in a specific environment (Hosie & Schibeci, 
2001 ). 


Evaluation checklists 

Checklists and evaluation forms have been criticised 
because of their focus on features that are external and 
easy to measure, not capturing the process of teaching 
and learning. Indeed, context-bound evaluation tools 
can actually cover a broader range of pedagogical 
issues because of the diversity of methodological tools 
used such as measurement of learning outcomes 
through tasks and assignments; conducting interviews 
with students and teachers, participant observation 
methods, collecting students’ work samples, video- 
taping students’ interaction, analysing students’ 
responses, and administering attitudinal scales (Hosie 
& Schibeci, 2001; Reeves & Harmon, 1994). 

Although context-bound strategies are powerful 
tools in bringing about a whole picture of the effec- 
tiveness of courseware, when it comes to evaluating a 
large quantity of educational material, such as the case 
of online resources, checklists do a faster job. This is 
particularly pertinent for teachers because of their job 
demands and constraints. Alessi and Trollip’s (2001) 
evaluation form builds on the framework of Alessi and 
Trollip’s (1991) quality review framework which 
addresses the evaluation of pedagogical features, 
interface design, navigation and user’s control of an 
online resource. 

Evaluation items for courseware 
(Alessi & Trollip, 2001) 

• Subject matter; 

• Auxiliary information; 

• Affective considerations; 

• Interface; 

• Navigation; 

• Pedagogy; 

• Invisible features; 

• Robustness; and 

• Supplementary materials. 


Evaluating websites 

The exploration of 500 mathe- 
matics education websites, using 
the above evaluation form, high- 
lighted some essential differences 
in design and usability issues 
between online resources and 
normal courseware. First, there is 
a diversity of online resource 
formats, namely: drills, tutorials, 
games, simulations, hypermedia- 
based materials and tools and 
open-ended learning environ- 
ments (Handal & Herrington, 
2003). Second, online resources 
differ from normal courseware in 
that the former do not come 
accompanied by a manual or 
printed instructions on how to 
teach with the resource. Finally, 
many online resources are 
embedded on webpages that are 
not consistent with other pages of 
the same website. As opposed to 
normal courseware, the organisa- 
tion and sequencing of online 
learning activities are not well 
articulated and goal-oriented 
making it difficult for teachers to 
choose especially when they are 
searching for activities supporting 
a specific curricular topic. 

The following section presents 
a summary of the important 
features identified through the 
evaluation of a large number of 
websites. 

Evaluation items for websites: 

• Introduction; 

• Displays; 

• Motivation; 

• Navigational aids; 

• Questions; 

• Self evaluation; 

• Content structure; 
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• Directions; 

• Learning metaphor; 

• Methodologies; 

• Format of feedback; 

• User control; 

• Language, style and grammar; 

• Help. 


Displays 

It is necessary to check whether (a) displays are 
uncluttered, (b) overwriting is avoided, and (c) atten- 
tion is maintained to relevant information. In terms of 
presentation, it is also important to review whether 
texts, graphics, colour and sound are used appropri- 
ately. Figure 2 shows a cluttered screen. 


Introduction 

Presentation of goals and objectives can enhance the 
understanding and motivational appeal of the subject 
matter and should be clearly stated and worded at the 
student’s lexical level. Information must be relevant, 
accurate and complete. Table of contents, indexes and 
directions must be clear and information must be 
accurate and related to the curriculum. The screen in 
Figure 1 provides students with ample information 
about the task. 


Ideas for Pupils 

Pupils working in small groups or independently 

U3« the thesaurus to check the meaning of malhemalic al terms used in your books or by your 
teachers The thesaurus will allow you lo look at terms or wards linked 1o your original enquiry. If 
you find ihst another term used in the definition you have found is not dear, you can use the 
reference® link lo find an explanation trflhis lerm 

You might also wish lo find out how the term links lo other concepts you hsrre met. The ihesaurus 
tries to map these connections for you 

It is possible to use the thesaurus when you sre reading other sites online. If you have installed the 
m-6utlon in your bruwssr's links bar you can use it to look up any mathematics word you may 
come across on any site 

How about using the thesaurus to prodice a map of you own around an area of mathematics you 
are currently studying? Are ihere ideas that you have not met before that look particularly 
interesting or connections that you had not thought about 7 You can produce a map of your own for 
display in the classroom Use the PilSc key lo capture maps 


You can use Ihe thesaurus lo help you find material on other malhemalics sites Most sites have a 
search function, but it can sometimes be tricky to know wtiat lo type m How do you find 
something whon you cant mmember wtial it is called? The answer rs to use the Ihesaurus firsli 
Start from something you know and use Ihe relations lo navigate lo the term you want. You can 
copy and paste Ihe term into a search engine - or use the ahoricuis to search engines thai we 
provide 


Motivation 

A webpage should maintain the user’s interest and 
must challenge the user across different displays. 
Visual momentum influences the learner’s ability to 
extract and absorb content that is relevant to him/her 
across successive displays. Features such as zoom, 
sound or animation must be assembled in unity and 
be consistent. Figure 3 shows a webpage with a 
dynamic percentage bar. 


Estimating Percentages 
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What percentage of the bar is shaded like this? 


Answer: 


% 


Check it! 


Score: 


Another? 


Figure 1. 

http://thesaurus. maths. org/mmkb/view.html?resource=guides 


Figure 3. http://www.hellam.net/maths2000/percent1.html 
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Navigation aids 

Tools availability should be checked to see whether 
the tools are active, or if they are present but are not 
active. Some tools should be removed or hidden from 
certain places. Otherwise, users get confused into 
thinking that the webpage is not working properly. 
For example, the control panel of a webpage might 
not be active in some sections. Most WWW browsers 
have sufficient navigational capabilities. Figure 4 
shows an easy to follow tool board for selection. 


Figure 2. http://pbskids.org/cyberchase/games/numbersense 
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— 

What's my angle? 

Use the buttons at the left to 
access ten online activities to 
support the teaching and 
investigation of angles. 

Teachers: 

Use the activities for 
demonstrating the use of 
protractors to groups. The 
activities are designed to be used 
through whiteboards and 
computer monitors. 

Pupils: 

Investigate and test yourself on 
the use of protractors. 


Copyright (c) Mark Robinson, 2000 


TEACHER CONTROLS 

Show the angle 
Make and Measure 
Make the angle game. 

Up to 90 degrees in tens 
Up to 180 degrees in tens 
j) Up to 90 degrees in fives 
^ Up to 180 degrees in fives 
Up to 180 degrees in ones 
Estimating up to 90. 



V 

EXIT 

- 


PLAY 

INTRO 


"\ 




Figure 4. http://ambleweb.digitalbrain.com/ambleweb/ambleweb/ 
ambleweb/mentalmaths/protractor.html 


Questions 

Questions should be relevant and be presented in a 
variety of formats. Likewise, the webpage must facili- 
tate learner’s answering by giving clear choices and 
the possibility of more than one try. Feedback must be 
relevant and supportive. Questions should be 
economical with instructions on answering questions. 
The activity on Figure 5 shows an activity linking 


Self-evaluation 

Self-evaluation can be achieved by giving the users a 
sense of accomplishment through acknowledgement 
or visual cues that indicate their progress. Self-evalua- 
tion can be achieved through, among others, self-tests 
or quizzes which require “yes” or “no” or multiple 
choice answers, or comments on results in simulation 
activity. The activity in Figure 6 provides continuous 
feedback on the task. 



Figure 6. http://www.aaamath.eom/B/addk7ex1.htm 

Content structure 

Menus should orient, give the opportunity of making 
a choice, and also of amending an incorrect choice. A 


numerical, graphical and symbolic data. 


dynamic menu is shown on Figure 7. 
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Figure 5. 

http://www.thelearningfederation.edu.au/repo/cms2/tlf/published/ 
1 0560/1 80204_education/L1 22_design_a_neighbourhood 
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Figure 7. 

http://www.bbc.co.uk/education/mathsfile/gameswheel.html 
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Directions 

Advance organisers assist learners in finding infor- 
mation. Providing the user with an overview of the 
topics to be covered and how to access them through 
hyperlinks in maps or menus is a good start for any 
webpage. A consistent method of using this informa- 
tion should be presented to the learner in the earlier 
stages with a on-screen reminder such as instructions. 
The screen on Figure 8 provides overview information 
about a webpage on symmetry. 


Understanding Congruence. Similarity, and Symmetry Using 
Transformations and Interactive Figures: Visualizing 
Transformations 



Vlui-lllzlllij iJtfUltr. .nil _iik»iiV.I ,u t.lju.linj . jmuusl'.u 


Thiiisfoiin.ntlons Tiani fcff nalibws -rtecflcij: .nnEfcrra.iGiis 

Rotations; translations, or slices, and reflections, or flips, are geometric transformations 
Liat tlialige an obiea's position oi onenlation but not its shape oi sizw ThelnleraUivv 
figures nttiistour-patTeYample allow a user to manipulate a shape and ohseive it? 
behavioi under a particulai transformation or composition or transfoi mation? Iri this 
part, Visualizing mansformstlons, on- can choose a transformation and apply it to a 
s:. apti to observe tfie resulting i mays In B w rev:t pail, idenlifying Uriknuwn 
T ranstoimations, the user s rhailangea to irlAliP/ the transformar on that has oeen 
used Iri Composing ons, users can examine tTi© 'esuitol refteciinga shape 
sucvessivelythroughlwo different lines And in the fourth part. Composing 
Transformations, the users aio ctiaKenged tu compose oquivsleol Lfansfomisconsiii 
two different ways Act vines like tnese allow stiioents to deepen rnei r understanding of 
congruence, si initially, and lefleamn. and ttiey also oonttlbule to Bre surety of 
transformations, as described mtne .e-me-iv ui Ciici 


Figure 8. 

http://standards.nctm.Org/document/eexamples/chap6/6.4/index.htm 



Figure 9. 

http://matti.usu.edu/nlvm/nav/frames_asid_1 97_g_2_t_1 .html? 
open=activities 


Learning metaphor 

The presentation of the information should be 
followed up by students’ activity, as students will be 
more motivated if they participate actively with the 
webpage. Also, learning experiences, when 
sequenced, must follow a specific theme or topic. The 
learning experience in Figure 9 relates to a collection 
of activities based on the number line bounce. 


Methodologies 

Student’s interaction with the webpage should be 
more proactive than reactive. A proactive interaction 
emphasises learner construction and generative 
activity whereas a reactive interaction is an answer to 
presented stimuli or to a given question. Interaction 
must be frequent and in a variety of forms. In Figure 
10 students are required to draw geometrical general- 
isations from manipulating objects. 


I TT 


home 
this month 

fwxl tier 
second tier 
nurd tier 
fourth tier 

ask NRICH 
thesaurus 
help 
courses 
maths 
finder 


■ai-Miiul Solution Printable page ! 

Rotating triangle 

Confer, 1 l ii vi 1 4 CMIIenpa Lav.- ★ 

Two smaller circles, centres B and C touch each other 
externally and touch the circle centre A internally. What 
happens to the perimeter of triangle ABC as the two 
smaller circles roll around touching the rim of toe bigger 
circle or as the two smaller circles vaty in size^ 

If you have a Java-enabled broker you can view an 
interactive version of this problem below. You can 
move toe red points around to ad|ust the diagram 
Moving the point X on the veto cal slider controls toe 
size of the smaller circle 




Figure 10. 

http://nrich.maths.org/public/viewer.php?obj_id=266&part=index 

&refpage=monthindex.php 

Format of feedback 

Appropriate webpages must consider the student’s 
awareness of his/her progress in the learning activity. 
A webpage should be organised in such a way that the 
amount of information does not overwhelm the user. 
Users should also know how the steps chosen are 
completed so that they can progress. The tutorial in 
Figure 11 provides step-by-step solutions for each 
problem. 
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Basic Proportion Works 1l 

.All -717^1 

Automatic Che clang 

Tins worksheet is enhanced with ai.it oma' 
Worksheet" at the bottom of each page. 


Figure 11. 

http://www.algebrahelp.com/lessons/proportionbasics/pgw.htm 


User control 

Control of the lesson is defined by the degree of 
command held by the learner over the webpage. 
Control includes navigation of the webpage, skipping 
the lesson, moving forward and backward and other 
interactions with the webpage. Likewise more control 
could be given for higher order thinking tasks such as 
problem solving and investigations in contrast to 
repetitive tasks. The webpage on Figure 12 allows 
users to choose the transformation they want to 
pursue. 



Language, style and grammar 

Language and grammar should be at the appropriate 
reading level. Technical terms and jargon should be 
avoided as much as possible while spelling and punc- 
tuation must be thoroughly edited. Figure 13 shows a 
high lexical density text. 


1.4 And At the Heart of Calculus Is ... 

The density property described above leads to a concept that is fundamental to calculus, and that is die 
concept of a limit If I gave you 2 gallons of milk today, one and a half gallons tomorrow, one and a 
tlui d gallons Hie next day, one and n fourth gallons the next day, and so on. and continued bi that 
manner for the rest of eternity, what can I say about how much milk you might get on a typical day? 
Well, theie is certamh a fomiula for it If I label the days, starting with today, as 1. 2. 3. 4, and so oil I 
can say dial on day number n, I will gh c yon 1 + 1 *'n gallons at milk But there is soiuetliuig stdl decpci 
I can sax- about all this. 1 can say that there is a lower bound on how much milk you'll get on any 
paiticnlar day That lowei bound applies to all days starting with today 

It you hied to argue that 1 01 gallons is a lowei bound. I could disprove it by noting that on the 101st 
day you would be getting less than that. But if you argued that any amount of one gallon or les s is a 
lower bound, I would be unable (o find an\ day m (lie future on winch 1 would be giving you less than 
that amount of milk So all amounts of one gnOou or less are lower bounds. But ot all of those, the 
amount of one gallon exactly is special. It is the greatest of all die lower bounds. And even though there 
is no day that 1 will give you exactly one gallon of null;, theie will come a da> on winch the amounr I 
give you will be a.s close to one gallon as I would like. Not only that, on all subsequent days the 
amount will be that close or closer 

hi odicr words, you tell me how close to one gallon I will be giving you. and I can name the dan,- on 

wluJi tin- ju ii n ml I di-hcn daih In vnu will hr furivrr a fin at Ii-aJ that t liter Thai is what inaLi-s 


Figure 13. http://www.karlscalculus.org/calc1.html 


Help 

A help function may be available for each task so that 
the learner has continuous guidance through the 
learning sequence as shown in Figure 14. 



Figure 12. http://www.waldomaths.com 


Figure 14. http://www.waldomaths.com 
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Conclusions and recommendations 

This paper dealt with issues associated with the inter- 
face design, navigation and user’s control of an online 
resource. It indicates how evaluation forms and check- 
lists can be practical tools for teachers to identify 
positive and negative design features of an online 
resource. The discussion also showed, in general 
terms, that the Alessi and Trollip’s (1991, 2001) frame- 
work can provide teachers with a simple and at the 
same time meaningful structure to assess WWW-based 
resources. These abundant resources require profes- 
sional judgment in their selection and articulation into 
the school mathematics curriculum. 

Generally speaking, it was found that online 
resources created by professional organisations and 
organised in inclusive websites such as the Learning 
Federation (www.thelearningfederation.edu.au), 
Cambridge University (www.nrich.maths.org), the 
National Council of teachers of Mathematics 
(illuminations.nctm.org/imath), York University 
(http://www.counton.org) or the Shodor Foundation 
(www.shodor.org), have a better instructional design 
than those created by individuals. These are compre- 
hensive websites whose online resources are more 
interactive, pedagogical oriented, sorted by grade level 
and curriculum objectives, thereby constituting a 
better search strategy for practicing teachers. 
Additionally, their URLs are also easier to remember! 
On the other hand, it is estimated that there are 500 
individuals’ websites — a figure that certainly reflects 
the growing enthusiasm and commitment of the math- 
ematics education community to produce and share 
resources using the WWW medium. Eventually some 
sort of centralised database of online resources by 
curriculum objective, grade level and/or type of appli- 
cation sought should be designed to facilitate teachers’ 
identification and access to the enormous amount and 
variety of online resources. The Teaching and 
Learning Exchange (TaLe) is a comprehensive educa- 
tional portal for parents, teachers and students 
developed by the NSW Department of Education and 
Training’s Centre for Learning Innovation. It provides 
access to a large range of resources that are organised 
by stages and by key learning areas. TaLe can be 
accessed at www.tale.edu.au. 


More research is certainly needed to modify course- 
ware evaluation instruments to the nature of online 
resources. Research is also needed to investigate the 
process of developing and supporting evaluation skills 
for practicing school teachers to facilitate the applica- 
tion of these worldwide resources in the mathematics 
classroom. 


References 

Alessi, S. & Trollip, S. (1991). Computer-Based Instruction: 

Method and Development. New Jersey: Prentice Hall. 

Alessi, S. & Trollip, S. (2001). Multimedia for Learning: Methods 
and Development ( 3rd Ed.), (pp. 410). Boston: Allyn & 

Bacon. 

Handal, B. & Herrington (2003). Re-examining categories of 
computer-based learning in mathematics education. 
Contemporary Issues in Technology and Teacher Education. 
Retrieved 10 July 2004 from 

http://www.citejournal.org/vol3/iss3/mathematics/articlel.cfm. 
Hosie, P., & Schibeci, R. (2001). Evaluating courseware: A need 
for more context bound evaluations? Australian Educational 
Computing, 16(2). 18-26. 

Reeves, T. C. & Harmon, S. W. (1994). Systematic evaluation 
procedures for interactive multimedia for education and 
training. In S. Reisman (Ed.), Multimedia Computing: 
Preparing for the 21st Centuty (pp. 472-505). Harrisburg, PA: 
Idea Group. 

Note 

A version of this paper was first published in M. 
Coupland, J. Anderson & T. Spencer (Eds) (2005), 
Making Mathematics Vital, Adelaide, AAMT Inc. 


Boris Handal 

(NSW DET) NSW Centre for Learning Innovation 
<boris.handal@det.nsw.edu.au> 

Parvin Handal 

NSW Western Sydney Health Area 
Tony Herrington 
University of Wollongong 
<tonyh@uow.edu.au> 

Ami 


14 APMC 1 1 (2) 2006 


